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■ Abstract 



ZJ ' Let P be a polytope with rational vertices. A classical theorem of Ehrhart states that the 

number of lattice points in the dilations P(n) = nP is a quasi-polynomial in n. We generalize 
this theorem by allowing the vertices of P{n) to be arbitrary rational functions in n. In this case 
we prove that the number of lattice points in P(n) is a quasi-polynomial for n sufficiently large. 
Our work was motivated by a conjecture of Ehrhart on the number of solutions to parametrized 
linear Diophantine equations whose coefficients are polynomials in n, and we explain how these 
' two problems are related. 

U' 

^ ! 1 Introduction. 

In this article, we relate two problems, one from classical number theory, and one from lattice point 
enumeration in convex bodies. Motivated by a conjecture of Ehrhart [EhrJ and a result of Xu [Xu], 
we study linear systems of Diophantine equations with a single parameter. To be more precise, 
we suppose that the coefficients of our system are given by polynomial functions in a variable n, 
and also that the number of solutions f(n) in nonnegative integers for any given value of n is 
^sO finite. We are interested in the behavior of the function f(n), and in particular, we prove that 

f(n) is eventually a quasi polynomial, i.e., there exists some period s and polynomials fi(t) 
for i = 0, . . . , s — 1 such that for t ^> 0, the number of solutions for n = i (mod s) is given by 
fi{n). The other side of our problem can be stated in a similar fashion: suppose that P(n) is a 
convex polytope whose vertices are given by rational functions in n. Then the number of integer 
points inside of P(n), as a function of n, enjoys the same properties as that of / as above. We now 
describe in more detail some examples and the statements of our results. 

1.1 Diophantine equations. 

As a warmup to our result, we begin with two examples. The first is a result of Popoviciu. Let a 
and b be relatively prime positive integers. We wish to find a formula for the number of nonnegative 
integer solutions (x, y) to the equation ax + by = n. For a real number x, let [x\ denote the greatest 
integer less than or equal to x, and define {x} = x — [x\ to be the fractional part of x. Then the 
number of such solutions is given by the formula 

+ 1, (1.1) 

where a -1 and 6 _1 satisfy aa^ 1 = 1 (mod b) and bb^ 1 = 1 (mod a). See |BRl Chapter 1] for a 
proof. In particular, this function is a quasi-polynomial in n. 
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For the second example which is a generalization of the first example, consider the number of 
solutions (x,y,z) G Z> to the matrix equation 

' mi | (1.2) 

m 2 

where the Xi and yi are fixed positive integers and Xi + \yi < xiyi + \ for i = 1,2. Write = 
Xiyj — xjyi. We assume that gcd(Yi2, *i3, 123) = 1, so that there exist integers (not unique) fij,gij 
such that 

gcd(/i 2 y"l3 + 012*23, *12) = 1, gcd(/i 3 yi2 + 013*23, *13) = 1, gcd(/ 2 3*13 + 023*12, *23) = 1- 

Now define two regions Oj = \(x,y) I — < - < for i = 1,2. Then if m = (mi,m.2) G Z 2 is in 

the positive span of the columns of the matrix in fjl .2 j) . there exist Popoviciu-like formulas for the 
number of solutions of (jl.2p which depend only on whether m G fii or m £ 1^2, and the numbers 
Yij, fij,g%j,Xi,yi- See Section [5TT1 for the precise statement. 

In particular, one can replace the Xi, yi, and rrii by polynomials in n in such a way that for all 
values of n, the condition gcd(Y"i2, *i3, Y23) = 1 holds. For a concrete example, consider the system 

2n + 1 3n + 1 n 2 \ ( X \ _ ( 3n 3 + 1 
2 3 n+iyl y )~ ^n 2 + n - 1 

Then for n>0, we have that 

3 3n 2 + n- l n + 1 
3n + 1 3n 3 + 1 n 2 ' 

so that for these values of n, there exists a quasi-polynomial that counts the number of solutions 

(x,y,z). 

Given these examples, we are ready to state our general theorem. We denote by QP >0 the set 
of functions /: Z — > Z which are eventually quasi-polynomial. 

Theorem 1.1. Let A(n) be an m x k matrix, and b(n) be a column vector of length m, such that 
their entries are integer coefficient polynomials in n. If f(n) denotes the number of nonnegative 
integer vectors x satisfying A(n)x = b(n) (assuming that these values are finite), then f G QP >0 - 

This theorem generalizes the conjecture [Stat Exercise 4.12]. See |Ehr[ p. 139] for some verified 
cases of a conjectural multivariable analogue, which we state here. Let S C Z r be some subset. We 
say that a function /: S — > Z is a multivariate quasi-polynomial if there exists a finite index 
sublattice LcZ r such that / is a polynomial function on each coset of L intersected with S. 

Conjecture 1.2 (Ehrhart). Let A(n±, . . . ,n r ) be an m x k matrix and b{n\, . . . ,n r ) be a column 
vector of length m, such that all entries are linear functions in m, . . . ,n r with integer coefficients 
such that for all n\, . . . ,n r , the number of nonnegative integer solutions x to A(nx, . . . ,n r )x = 
b(ni, . . . , n r ) is finite. Then there exist finitely many polyhedral regions Ri, . . . , Rn covering R> 
such that f is a multivariate quasi-polynomial when restricted to each Ri . 
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1.2 Lattice point enumeration. 

We first recall a classical theorem due to Pick. Let P C R 2 be a convex polygon with integral 
vertices. If A(P), I(P), and B{P) denote the area of P, the number of integer points in the 
interior of P, and the number of integer points on the boundary of P, respectively, then one has 
the equation 

A(P) = I(P) + \B{P) - 1. 

Now let us examine what happens with dilates of P: define nP = {nx j x 6 P}. Then of course 
A(nP) = A(P)n 2 and B(nP) = nB(P) whenever n is a positive integer, so we can write 

A(P)n 2 = I(nP) + ^B(P)n - 1, 

or equivalently, 

#(nP n Z 2 ) = I(nP) + B(nP) = A(P)n 2 + ^B(P)n + 1, 

which is a polynomial in n. The following theorem of Ehrhart says that this is always the case 
independent of the dimension, and we can even relax the integral vertex condition to rational 
vertices: 

Theorem 1.3 (Ehrhart). Let P C R rf be a polytope with rational vertices. Then the function 
Lp(n) = #(nP n Z d ) is a quasi-polynomial of degree dim P. Furthermore, if D is an integer such 
that DP has integral vertices, then D is a period of Lp(n). In particular, if P has integral vertices, 
then Lp{n) is a polynomial. 

Proof. See [Sta] Theorem 4.6.25] or [BR S Theorem 3.23]. □ 

The function Lp{t) is called the Ehrhart quasi-polynomial of P. One can see this as saying 
that if the vertices of P are Vi = (vn, . . . then the vertices of nP are given by the linear 

functions Vi(n) = (van, . . . ,Vid,n). We generalize this as 

Theorem 1.4. Given polynomials Vij(x),Wij(x) € Z[x] for < i < s and 1 < j < d, let n 
be a positive integer such that Wij(n) ^ for all i,j. This is satisfied by n sufficiently large, 
so we can define a rational polytope P(n) = coiw(p°(n),p l (n), . . . ,p s (n)) € H d , where p l {n) = 

■ ■ ■ ' 22$)" Th ™ #tnn) n Z<) E QP >>0 . 

We call the function ^{P{n) n Z d ) a generalized Ehrhart polynomial. 

In Section [21 we explain the equivalence of the two problems just mentioned, then prove our 
main result Theorem II .ll in Section [3l The proof gives us an algorithm to compute these generalized 
Ehrhart polynomials, but it could be very complicated in practice. For computational reasons, we 
introduce the notions of generalized division and generalized gcd for the ring Z [x] in Section HI In a 
sense, these generalize the usual notions of division and gcd for the ring of integers Z. The methods 
and algorithms are quite similar, but are more involved due to technical complications. The proofs 
for the correctness of the algorithms are given in this section. As an application of these tools, in 
Section [5j we will describe explicit computations of some special generalized Ehrhart polynomials. 
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2 Equivalence of the two problems 



As we shall see, the two problems of the Diophantine equations and lattice point enumeration are 
closely intertwined. In this section, we want to show that Theorem 11,11 is equivalent to Theorem 
11,41 Before this, let us see the equivalence of Theorem 11.41 with the following result. For notation, 
if x and y are vectors, then x > y if Xi > for all i. 

Theorem 2.1. For n ^ 0, define a rational polytope P(n) = {x G R rf | V{n)x > c(n)}, where V 
is an r x d matrix, and c is an r x 1 column vector, both of whose entries are integer coefficient 
polynomials. Then #(P(n) n Z d ) G QP >0 

Notice that the difference of Theorem 11.41 and Theorem 12.11 is that one defines a polytope 
by its vertices and the other by hyperplanes. So we will show their equivalence by presenting a 
generalized version of the algorithm connecting "vertex description" and "hyperplane description" 
of a polytope. 

The connection is based on the fact that we can compare two rational functions f(n) and g(n) 
when n is sufficiently large. For example, if f(n) = n 2 — An + 1 and g(n) = 5n, then f{n) > g(n) 
for all n > 9, we denote this by /(n) > CV cn <?("-) ("even" being shorthand for "eventually"). 
Therefore, given a point and a hyperplane, we can test their relative position. To be precise, 
let p(n) = (ri(n), . . . , rfc(n)) be a point where the rj(re) are rational functions and let F(x,n) = 
ai(n)x\ + a2(n)x2 + • • • + ak{n)xk = be a hyperplane where all the aj(n) are polynomials of n. 
Then exactly one of the following will be true: 

F(p, n) =even 0; F(p, n) > even 0; F(p, n) < even 0. 

Given this, we can make the following definition. We say that two points p(n) and q(n) lie (resp., 
weakly lie) on the same side of F(p,n) if F(p,n)F(q,n) > eve n (resp., F(p,n)F(q,n) > even 0). 

2.1 Equivalence of Theorem 11.41 and Theorem 12.11 

Going from the "vertex description" to the "hyperplane description" : 

Given all vertices of a polytope P(n), whose coordinates are all rational functions of n, we want 
to get its "hyperplane description" for n S> 0. Let F(x, n) be a hyperplane defined by a subset 
of vertices. If all vertices lie weakly on one side of F(x,n), we will keep it together with > 0, or 
< or = indicating the relative position of this hyperplane and the polytope. We can get all the 
hyperplanes defining the polytope by this procedure. 

Going from the "hyperplane description" to the "vertex description": 

Let .P(n) = {x 6 JH d \ V(n)x > c(n)} be a polytope, where V is an r x d matrix, and c is an 
r x 1 column vector, both of whose entries are integer coefficient polynomials. We want to find its 
vertex description. Let /i(n), . . . , f r (n) be the linear functionals defined by the rows of V(n). So 
we can rewrite P{n) as 

P(n) = { x G n d | (fi(n),x) > a{n) for all i}. 

The vertices of P(n) can be obtained as follows. For every ci-subset / C {1, . . . , r}, if the equations 
{{fi(n),x) = Ci(n) | i G /} are linearly independent for n ^> 0, and their intersection is nonempty, 
then it consists of a single point, which we denote by vi(n). If {fj(n),vi(n)) > Cj(n) for all j, then 
vi(n) G Q(n) rf is a vertex of P(n), and all vertices are obtained in this way. We claim that the 
subsets / for which vi(n) is a vertex remains constant if we take n sufficiently large. First, the 
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notion of being linearly independent equations can be tested by showing that at least one of the 
d x d minors of the rows of V(n) indexed by I does not vanish. Since these minors are all polynomial 
functions, they can only have finitely many roots unless they are identically zero. Hence taking 
n ^> 0, we can assume that {fi(n) \ i € 1} is either always linearly dependent or always linearly 
independent. Similarly, the sign of (fj(n),vi(n)) is determined by the sign of a polynomial, and 
hence is constant for n>0. 

2.2 Equivalence of Theorem 11.11 and Theorem 12.11 

We can easily transform an inequality to an equality by introducing some slack variables and we 
can also represent an equality /(n, x) = by two inequalities /(n, x) > and —f(n, x) > 0. So the 
main difference between the two theorems is that Theorem 11.11 is counting nonnegative solutions 
while Theorem 12. II is counting all integral solutions. But we can deal with this by adding constraints 
on each variable. 

A more interesting connection between Theorem 1 1 . 1 1 and Theorem 12. II is worth mentioning here. 
First consider any fixed integer n. Then the entries of A(n) and b(n) in the linear Diophantine 
equations A(n)x = b(n) of Theorem II .11 all become integers. For an integer matrix, we can calculate 
its Smith normal form. Similarly, we can use a generalized Smith normal form for matrices over 
QP >0 to get a transformation from Theorem 11.11 to Theorem 12.11 

Then given A{n) and b(n), by Theorem 14. 6} we can put A(n) into generalized Smith normal 
form: D{n) = U{n)A{n)V{n) for some matrix 



with nonzero entries only on its main diagonal, and unimodular matrices U(n) and V(n). Then 
the equation A(n)x = b(n) can be rewritten as D(n)V(n)~ 1 x = U(n)b(n). Set y = V(n)~ 1 x and 
b'{n) = U{n)b{n). By the form of D(n), we have a solution y if and only if di{n) divides b'^in) for 
i = 1, . . . ,r, and for any given solution, the values y r +i, ■ ■ ■ ,Uk can be arbitrary. However, since 
V(n)y = x, we need to require that V(n)y > 0, and any such y gives a nonnegative solution x to the 
original problem. Simplifying V{n)y > 0, where V(n) = (vi(n), . . . ,Vk(n)), we get V'{n)X > c(n), 

where V'{n) = (v r+1 (n), . . . , v k (n)), X = (y r+1 , . . . ,y k ) and c(n) = -(t?i(n)yi H + v r (n)y r ). 

Although V'(n) and c(n) has entries in QP >0 , we can assume that they are polynomials by 
dealing with each constituent of the quasi-polynomials separately. So we reduce Theorem 11.11 to 
Theorem 12.11 

Example 2.2. Consider the nonnegative integer solutions to 



D(n) = (diag(di(n) J ...,d r (n),0,...,0)|O) 



(n 2 + 2n 




(2.1) 




2n + 2) 



When n 



2m, the Smith normal form of A(2m) is 




0). 



So the equation (|2.ip becomes 
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Set ( , N | ( Xl ) = ( yi I, so that we have 



(4m + 4 0) fa\ = (2n + 4) (n 2 + 2n) , 



and thus y\ = n 2 + In. By the condition ( Xl I > 0, we require 



1 -l\ (n 2 + 2n 



-(m — 1) m J \ 2/2 



> 



which gives us 



(m - l)(n 2 + 2n) 9 

— < Wi < n 2 + 2n, 



rn 

a one dimensional polytope. So the number of solutions for n = 2m is f(n) = 2n + 5 
We can do the case n = 2m + 1 similarly. The Smith normal form of A(2m + 1) is 

(4m 2 + 8m + 2 4m + 6) ( 1 " 2 i ]=(2m + 3 0) 
v ' \—m 2m + I J v ' 

an d 2 " 1 2m+i 2W ' > — ^ 2 — n2 + tnus = n + 3. 

The proof of Theorem 14.61 is based on a theory of generalized division and GCD over the ring 
Zi[x], which mainly says that for f(x), g(x) £ Z [x] , the functions , |^y|, and gcd(/(n), (ji(n)) 

lie in the ring QP >0 . One interesting consequence of these results is that every finitely generated 
ideal in QP^o ^ s principal, despite the fact that QP > o ^ s n °t Noetherian. We developed this theory 
in order to appoach Theorem II .11 at first, but subsequently have found a proof that circumvents its 
use. Further details can be found in Section [U 



3 Proof of Theorem 11.11 

To prove Theorem II .1\ we will use a "writing in base n" trick, to reduce equations with polynomial 
coefficients to linear functions. Briefly, the idea of the following "writing in base n" trick is as 
follows: given a linear Diophantine equation 

ai(n)xi + a 2 (n)x 2 H h a k (n)x k = m(n) 

with polynomial coefficients (ii(n) and m(n), fix an integer n, then the coefficients all become 
integers. Now consider a solution {x\, X2, ■ ■ ■ , x k ) with x% G Z>o- Put the values of (xi, X2, ■ ■ ■ , x k ) 
into the equation, then both sides become an integer. Then we use the fact that any integer has a 
unique representation in base n (n is a fixed number), and compare the coefficient of each power 
of n in both sides of the equation. 

Finally, letting n change, we happen to have a uniform expression for both sides in base n when 
n is sufficiently large. Moreover, the coefficient of each power of n in both sides of the equation 
are all linear functions of n (Lemma 13.11 and Lemma 13 . 2[) . Then by Lemma 13. 3\ we can reduce 
these equations with linear function coefficients to the case when we can apply Ehrhart's theorem 
(Theorem 1 1 . 3|) to show that the number of solutions are quasi-polynomials of n. This completes 
the proof of Theorem 11.11 
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Lemma 3.1. Given p(x) € Z[x] with p(n) > for n S> (z.e., p(x) has positive leading coefficient), 
there is a unique representation of p(n) in base n: 

p{n) = c d (n)n d H h c a (n)n + c (n), 

where Cj(n) is a linear function of n such that for n>0, < Cj(n) < n — 1 /or i = 0, 1, . . . , d and 
< Cd{n) < n — 1. VFe denote d = deg n (p(n)). 

Note that deg n (p(n)) may not be equal to deg(p(n)). For example, n 2 — n + 3 is represented as 
c\{n)n + co(n) with d = 1, ci(n) = n — 1, and co(n) = 3. 

Proof. Let = o^x^ + • • • + aix + ao with integral coefficients and > 0. If the aj are all 
nonnegative, the proposition holds for n > maxo<j<^{aj}, otherwise we can prove it by induction 
on i — i, where i is the smallest index such that ai < 0. Suppose io € {0, ...,£ — 1} is the minimal 
number such that ai < 0, then in the representation of p(n), put Cj = aj for j < io, a = n — ai . 
By induction, we have a representation for 

p(n) - (n + a io )n 4 ° - a^-in 10 ' 1 a , 

so we can add it to (n + ai )n l ° + • • • + ao to get the desired representation for p(n). Since for n ^ 
we have Cj = n + a, > 0, by induction, we can make sure < Cj < n — 1 for n>0. For a lower 
bound, C = maxo<i<£{|aj|} + 1 is sufficient, i.e., for all n > C, the desired unique representation is 
guaranteed to exist. Note that io ^ t, since ag > 0. So this process will stop in finitely many steps. 
Uniqueness of this representation is clear. □ 

Lemma 3.2. Fix an integer N and n ^> 0. Consider the set 

Si(n) = {(a?i, . . . , Xk) G Z fc | < Xj < n N+1 , ai(n)xi + a2{n)x2 + • • • + ak(n)xj. = m(n)} 

where aj(n) = X^=o a *^ n ^ ( as a usua ^ polynomial) and m(n) = X^=o (represented in base n as 
in Lemma \3. 1}) . with bd ^ and d > maxi<j<fc{dj}. T/ien S'i(n) is in bisection with a finite union 
of sets of the form 

£2 = {0 < (xjj) i<j<fc < n,Xjj € Z j all constraints on x = (x{j) are of the form An+B = f(x)} 
o<j<N 

where A, B G Z and f(x) is a linear form of x with constant coefficients. 

Proof. Fix a sufficiently large positive integer n. By our assumptions, for any point [x\, . . . ,Xk) € 
S±(n), we can write Xi = Xi t wn N + a^.TV-in^ -1 + • • • + x^o with < Xij < n. The rest of the 
lemma is a direct "base n" comparison starting from the lowest power to the highest power in the 
equation ^aj(n)xj = m(n). To get a feel for the proof, we recommend that the reader look at 
Example 13.51 first. We will use this as a running example to explain the steps of the proof. 

First write m(n) = mN(n)n N + m^-i (n)n N ~ 1 + • • • + mo{n) in base n, so that each mi(n) = 
m"n + m! i is a linear function in n. Then we know that < rrii(n) < n since we have fixed n 
sufficiently large. Going back to Example 13.51 we have m(n) = 4n 2 + 3n — 5 so that ni2(n) = 4, 
mi(n) = 2 and mo(n) = n — 5, so that here "sufficiently large" means n > 5. Now expand out the 
equation Yl, a i{ n ) x i = Tn(n) to get 

k / <k \ 

^ (52 aan e )(x i)N n N H h x i)0 ) = m N {n)n N H h m (n). (3.1) 

i=l \ £=o / 
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The constant term in base n of (|3.ip gives us the equation 

aio^i.o H h afco^A;,o = m (n) (mod n). 

Since we have the bound < x^q < n for all i, we can in fact say that the LHS is equal to 
m o( n ) + Cqti where Cq is an integer such that Co is strictly greater than the sum of the negative 
ajo and strictly less than the sum of the positive (and Co can also be 0). But note that Co only 
depends on the a,o, so if our n is sufficiently large, we may assume that |Co| < n. Going back to 
Example 13.51 we have the equation 

2xio + %20 = n — 5 (mod n), 

so that < C < 2. 

So now we have finitely many cases for the value of Co to deal with. Fix one. Going back to 
the equation (|3.1|) . we can substitute our value of Co and compare linear coefficients to get 

k 

y~](<HiXi,o + aioXi,i) + C = rai(n) (mod n). 
i=i 

Again, since we know that < x^o < n and < x^i < n for all i, we can say that the LHS is equal 
to mi(n) + C\n where C\ is an integer such that C\ is greater than the sum of the negative a^o 
and an and less than the sum of the positive c^o and an (and this is independent of Co because 
| Co | < n). Going back to Example 13. 5} we get the equation 

2xn + X21 + x 2 o + Co = 2 (mod n), 

so < Ci < 4. 

Now we have finitely many cases of Ci, and again we fix one. We continue on in this way to get 
C2, C3, . . . , Cv-i- At each point, we only had finitely many choices for the next Ci, so at the end, 
we only have finitely many sequences C, = (Co, . . . , Cjv-i)- For a given C,, we see that the Xij 
must satisfy finitely many equalities of the form f(x) = An + B where / is a linear form in x with 
constant coefficients and A, B £ Z. In Example 13.51 f° r C» = (0) 2), we would have the equations 

2xio + X20 = n - 5, 
2xn + X21 + X20 = 2 + 2n, 
2xi2 + x 2 i + x 30 + 2 = 4, 

along with the inequalities < Xjj < n, and we have underlined the places where C\ appears. 

Each such C, gives us a set of the form S2 (n) , and to finish we take the union of these sets over 
all sequences C.. □ 

We will need one more prepatory lemma before doing the proof of Theorem 11.11 

Lemma 3.3. If P(n) is a polytope defined by inequalities of the form An + B < f(x), where 
A,B^Z and f(x) is a linear form of x with constant coefficients, then Lp € QP >0 . 

Proof. We first use the fact that the combinatorics of P{n) stabilizes for sufficiently large n (see 
Section [2]), say that the polytope has s vertices v\(n), . . . ,v s (n). Also, note that the coefficients 
of the vertices of the polytope are linear functions in n with rational coefficients. Let D be the 
least common multiple of the denominators that appear in all of the coordinates. Then by [Bar . 
Theorem 18.4], the function sending (v\(n), . . . ,v s (n)) to the number of integer points in P(n) is 
a polynomial if we restrict to a specific congruence class of n modulo D. We can compose each of 
these functions with the polynomial n h-> (vi(n), . . . ,v s (n)) to conclude that Lp E QP >0 . □ 
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We can also give a self-contained proof of Lemma 13.31 that only uses Ehrhart's theorem (Theo- 
rem [L3]). 

Alternate proof of Lemma \3.3[ We will do induction on dimension first, the case of dimension 
being trivial. An inequality of the form An + B < f(x) with B ^ will be called an inequality 
with constant term. Secondly, we will do induction on the number of inequalities with constant 
term. Theorem 11.31 gives the desired result when there are no inequalities with constant term. In 
general, let P{n) be a polytope defined by An + B < f(x) plus other relations of the same form, 
which we will call R(n). Let P'(n) be the polytope with relations An < f(x) and R(n). If B > 0, 
then for i = 0, . . . , B — 1, let Pi(n) be the polytope with relations An + i = f(x) and R(n). Then 

B-l 

L P (n) = Lp<(n) - L Pi (n), 

so to show Lp G QP >0 , it suffices to show that Lpi € QP>o an d L Pi € QP>o for i = 0, . . . , 5 — 1. 
Since P'(n) has one less inequality with constant term, we know that Lp> G QP^q by induction. 
As for Pi(n), write f(x) = c\X\ + • • • + c^Xk and choose j such that Cj ^ 0. Then the equation 
An + i = f{x) can be rewritten as 

Xj = cJ 1 (An + i — c t x i)- 

Making substitutions into the relations R, each relation still has the form A'n + B' < f'(x) (clear 
denominators if necessary to make sure all of the coefficients are integral), and we have eliminated 
the variable Xj. 

If B < 0, then for i = 1, . . . , —B, let Pi(n) be the polytope with relations An + i = f(x) and 
R(n). Then 

-B 

L P (n) = L P /(n) + ^L Pi (n), 

i=l 

and we proceed as before. Hence Lp i £ QP>o by induction on dimension. □ 

Proof of Theorem \l.l\ Consider a system of linear Diophantine equations A{n)x = b(n) with 
finitely many nonnegative integer solutions for each n > 0, as in Theorem 11.11 Let 

ai(n)x\ + a2(n)x2 + ■ ■ ■ + ak(n)xk = m(n) 

be any equation from the system. The equations A(n)x = b(n) define a bounded polytope whose 
vertices are given by rational functions of n as discussed in Section [21 where the degrees of the 
rational functions (difference between the degree of the numerator and the denominator) are inde- 
pendent of n. We can express each point in this polytope as a convex combination of the vertices. 
Therefore, writing each variable x\, . . . in base n, we can find some positive integer N, which 
does not depend on n, such that the coordinates of each point in the polytope are less than n N+1 . 
Using Lemma 13.21 we can reduce each equation from A(n)x = b(n) into a new system like 52, with 
more variables than Si but all restrictions are of the form An + b < f(x). Applying Lemma 13.31 
finishes the proof. □ 
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3.1 Examples. 

Example 3.4. We give an example for Lemma 13.31 For n a positive integer, let P(n) be the 
polygon denned by the inequalities x > 0, y > and —2x — y > —n — 1. Then P'{n) is defined by 
the inequalities x > 0, y > 0, and 2x + y < n, and -Pi(n) is defined by the inequalities x > 0, y > 0, 
and n + 1 = 2x + y. We can rewrite the equality as y = n + 1 — 2x, and then the other inequalities 
become x > and n + 1 > 2x. 

We see that P'(n) is the convex hull of the points {(0, 0), (0, n), (n/2, 0)}, while P\{n) is the 
interval [0, (n + l)/2]. The total number of integer points in P'{n) and P\{n) is given by the 
quasip olynomial 

jk 2 + 3k + 2 \in = 2k 
) k 2 + 4k + 4 if n = 2fc + 1 ' 



#(P(n)nz 2 ) 

Its rational generating function is 



V MPM n 7 2 V™ - ^ 5 -3t 3 + 4t + 2 _ t 3 - 2t 2 + 2 

„t/ ( ()n ' " (l- t 2)3 -(l- t )3(l + t) - 

Example 3.5. We give an example of Lemma 13.21 Consider nonnegative integer solutions for 

2x\ + (n + 1)^2 + ^ 2 X3 = 4n 2 + 3n — 5. 

For any n > 5, RHS = 4n 2 + 2n + (n — 5) is the expression in base n. Now consider the left hand 
side. Writing x\, X2, X3 in base n, let xi = xun 2 + xnn + xio, X2 = X2\n + X20 and X3 = X30 with 
< Xij < n. In this case, N = 2, but we have further restrictions on the degrees of x<i and X3 in 
base n coming from the coefficients a(n). Then we have 

LHS = (2xi2 + %2l + X3o)n 2 + (2xu + x 2 i + X2o)n + (2a?io + ^20)- 

Now we can write the left hand side in base n with extra constraints on (xij)'s. 

We start with comparing the coefficient of n° in both sides. We have the following three cases: 

^0 = {2xio + X20 = n - 5}, 

A? = {2xio + x 20 = (n - 5) + n}, 

and 

A° 2 = {2xio + £20 = (n - 5) + 2n}. 

So in the language of proof of Lemma 13.2^ we have < Co < 2. We next consider n 1 term. If x 
satisfies A® Cq for n°, Co 6 {0, 1, 2}, then the equation is reduced to 

(2xi2 + £21 + x 30 )n 2 + (2xn + x 2 i + x 2 o + C )n = An 2 + 2n. 

Now compare the n 1 terms. We have five cases for each Co € {0, 1, 2}: 

^CoA = i 2x n + ^21 + x 20 + C = Cin + 2}, 

where Ci G {0,1,2,3,4}. 

Last, we compare the n 2 terms. Note that since we assume n>0, the n° term won't carry over 
to n 2 term, so the computation of n 2 term only depends on the term n 1 . If x satisfies the Cith 
condition for n , the equation then becomes 



(2xi2 + £21 + x 30 + Ci)n 2 = 4n 



2 
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So for each < C\ < 4, we have 

A 2 Ci = {2xi2 + xai + £ 30 + Ci = 4 >- 

Overall, we have the set 

{(xi, X2, X3) G Z> I 2xi + (n + l)x2 + n 2 X3 = An 2 + 3n - 5} 
is in bijection with the set 

{x = (xi2,xn,xio,2;2i,2;2o,X3o) G Z| ,0 < Xij < n}, 
such that x satisfies the equations in the sets 

(^00 An 
^10 ^11 
A 1 A 1 
^20 ^21 

Here we borrow the notation of matrix multiplication AB to represent intersection of sets An B 
and matrix summation A + B to represent set union AuB. Note that here all constrains A\ on 
x = (X12, xn, a?io, X21, X201 £30) are in the form of An + < /(x), where 4,B £ Z and f(x) is a 
linear form of x with constant coefficients. 




4 Generalized division. 

An integer- valued polynomial, or a numerical polynomial is a function / : Z — > Z with a 
polynomial g(x) G Q[x] satisfying /(n) = g(n) for all n G Z. For any numerical polynomial / and 
a constant c 6 Z, it is not hard to see that |_^7^J G Qf>o- We want to generalize this to 
for two numerical polynomials / and g. Based on classical division, we have for all n G Z that 
f( n ) = g(n7 + |f(ny}^^ n )' wr ^h ^ — |f[ny}^( n )' we define two integer- valued functions 

p(n) = , called the quotient of f(n) and g(n) and r(n) = |^^y j 9( n )i called the remainder 

of f(n) and g(n). We will show that p,rG QP >0 by the generalized division algorithm. 

This generalized division algorithm is based on the division for /, g G Q [x] , except for two 
modifications. First, in order to have < r[n) < g(n) for n S> 0, we need to change the remainder 
condition of division in Q[x], as in [Algorithm [TJ lines I28H33| . For example, for f(x) = x — 3 and 
g(x) = x, then we should have p(x) = 0, r(x) = x — 3, but these do not satisfy deg(r) < deg(g) 
nor r = for division in Q[x]. Secondly, in order to keep p(x),r(x) G Z[x], we break x into finitely 
many "branches" whenever we have noninteger coefficients, as in [Algorithm [H lines [TlTl20] . For 
example, take /(x) = x 2 + 3x and g(x) = 2x + 1. Trying to do division in Q[x], leading coefficient 
of p will not be an integer, so we break x into 2 "branches", since 2 is the leading coefficient of g(x). 
Now we do division to f(2t) = 4t 2 + 6t and g(2t) = At+1, when x = 2t; and /(2i+l) = At 2 + 10t + A 
and g(2t + 1) = At + 3, when x = 2t + 1. For x = 2t, we have po(t) = t + 1 and ro(i) = t — 1 
and for x = 2t + 1 we have pi(t) = i + 1 and r\(t) = 3t + 1. We will show (see Theorem 14. ip that 
breaking x into T = (using notation in Algorithm [1]) "branches" is sufficient to conclude that 
Pi(t),n(t) GZ[t],fori = 0,...,T-l. 

Note that in Algorithm [H we define lc(r) to be the leading coefficient of r. 

Theorem 4.1. Algorithmic is correct. 
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Algorithm 1 Generalized division for r L\x\. 



Input: f(x),g(x) € Z[x]. 



f(x) = a k x k + a fc _ix fc 1 H h aix + a 



gr(x) = 6^ + bg-ix* 1 H h bix + b 

Output: T and Pi(x), ri{x) € Z[x], for i = 0, . . . , T — 1 such that 



Pi(m) 



9(n) 



and ri(m) 



fin) 

g(n) 



g(n), where n = Tm + i and n S> 



1 


if k < i then 




2 


output T 


= 1 




3 


if a,k > then 


4 


p(x) = 


0; r(x) = f{x). 


5 


else if a^. < 


and 6^ < then 


6 


p(x) = 


1; r (x) = fix) - gix). 


7 


else if a,k < 


and bi > then 


8 


p(x) = 


-1; r(x) = /(x) + #(x). 


9 


end if 






10 


else 






11 


output T 






12 


for all i 


from to T - 1 do 


13 


Set ri(x) = /(Tx + i); pi(x) = 


14 


while K > 


g do 


15 


h(x) 


b e T eX ' 


16 


ri(x) 


= J"j 


(x) - hix)giTx + i 


17 


Pi(x) 


= Pi 


(x) + /i(x); 


18 


K = 


deg(ri); 


19 


a = lc(rj) 




20 


end while 




21 


while 


K = 


g do 


22 


c = 


a 




23 


Pi{x) 


= Pi 


(x) + c; 


24 


Ti(x) 


= n 


(x) — cg(Tx + i) 


25 


a = lc(rj) 




26 


K = 


deg(r-j) 


27 


end while 




28 


if a > then 


29 


Output Ti{ 


x), pj (a;); 


30 


else if 


a < then 


31 


n{x) 


= J"j 


(x) + #(Tx + i); 


32 


Pi(x) 


= Pi{x) - 1 


33 


end if 






34 


end for 






35 


end if 
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Proof. The algorithm will stop after finitely many steps (deg(r) is reduced by 1 each step in [line 
116]). For i = 0, . . . , T — 1, and for n = i (mod T), we claim the following two properties: 

1. ri(x),pi(x) G Z[x]. 

2. f(Tx + i) = pi(x)g(Tx + i) + ^(x) and < 7j(m) < ^(n), for n ^ and n = Tm + i. 

These two properties imply the correctness of the algorithm. By the algorithm, [2] is obvious, so we 
only need to showCO Note that in each step, we add to Pi{x) some h{x) = ]^fiX K ~ e or an integer c 
(see [lines [T71 and [23] ) . and subtract h(x)g(Tx + i) or cg(Tx + i) from rj(a?) (see [lines [TBI and 124"] ) , 
so it suffices to show h(x) G Z[x\. For every ri{x) with degree X in [lines [TT1 - [20] . denote it by 
rf{x) and its leading coefficient by or-. So we need to prove that biT e \ ax for all K > i. To show 
this, we use induction on s = k — K to prove a stronger statement that T d | coeff (x d ) for every 
monomial x d in rf(x). If s = 0, as in [line[13], rf~(x) = f(Tx + i) and the claim holds by binomial 
expansion. Assume it still holds for some s such that K = k — s > i + 1. Then for s + 1, we have 
K = k-s- 1. In pineEBJ, 

rf ( x ) = rf +1 (x) - h(x)g{Tx + i), 

so we only need to show that 

h(x)g(Tx + i) = ^x k - M g{Tx + i) 

also has the property that T d \ coeff (x d ), for every monomial in h(x)g(Tx + i). This is true because 
T d | coeff (x d ) in g(Tx + i), and in /i(n) we have T k ~ s \ a^-s by hypothesis and bgT^ \ T i+1 for 
b e | T, and thus T k ~ s -( i+1 ) \ . □ 

We want to generalize Algorithm [1] to QP >0 . First we need another expression for numerical 
polynomials. 

Lemma 4.2. Let f be a numerical polynomial, i.e., f: Z — > Z with a polynomial g(x) G Q[x] 
satisfying f(n) = g(n) for all n G Z. Then there exists T, and g%(x) G Z,[x] for i = 0, ... ,T — 1 
suc/i i/iai /(n) = gi(m) for all n = Tm + i. 

Proof. Suppose Tg(x) G Z[x] for some integer T. For i = 0, . . . , T — 1, define <?i(cc) = g(Tx + i). By 
the binomial expansion of g(Tx + i) — g(i), it is easy to see that this is a polynomial with integer 
coefficients. □ 

Similarly, for an element / G QP >>0 , there exists T G N and hi(x) G Z[x] (i = 1,...,T), 
satisfying f(n) = hi(m) for n S> and n = Tm + i. We call / = (T, {/ij(x)}^l 1 ) a representation 
°f / € QP >0 . Another simple property is that for / = (T\, {fi(x)}J=i) and g = (T2, we 
can extend to a common period T = lcm(Ti,T2) and / = (T, {fl(x)}f =1 ) and <? = (T, {^(ic)}^!)- 

Theorem 4.3 (Generalized GCD). Let f,g be numerical polynomials, or more generally f,g G 
QP >0 . Define d{n) = gcd(/(n), <?(n)). T/ien 6?/ t/ie generalized division over QP > o (Algorithm 
\^), we have d G QP >0 - ca ^ ^ ^ e generalized gcd 0// and g, and denote it by ggcd(/, g). 

Proof. Apply Algorithm [2] successively. Define r_2 to be some constituent polynomial of / and r_ 1 
to be some constituent polynomial of g (a constituent polynomial of a quasi-polynomial is a 
polynomial on a congruence class modulo its period). In general, if rj_i 7^ 0, define rj to be some 
constituent polynomial of rem(rj_2> ^i-i)- We will prove that = for some z by induction on 
deg(r) and lc(r) (Recall that lc(r) means the leading coefficient of r.). Hence, since our choice of 
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Algorithm 2 Generalized division over QP 



>o- 



Input: f,g G QP >>0 

Output: T and p%(x), r,(x) G Z[z], for i = 0, . . . , T — 1 such that 



Pi(m) 



/(") 



and n(m) 



fin) 

g{n) 



g(n) where n = Tm + i and n ^> 



1: Find a common period T for f,g, such that / = (T , {/j^)}^) and 5 = (T , {5i(^)}i=i)- 

2: for i from to To — 1 do 

3: if gi(x) = then 

4: Stop. 

5: else 

6: Apply Algorithm 1 to fi(x) and gi(x), get T, (x), r[. (x) G Z[x], j = 0, . . . , T - 1. 

7: Set T = Tq lcm(Tj [ £ = 0, . . . , To — 1), and get Pi(x), r; L (x) G Z[x], for i = 0, . . . , T — 1 from 

8: end if 

9: end for 



constituent polynomial was arbitrary, we will eventually have a remainder of after finitely many 
steps. Hence the algorithm terminates after finitely many steps, or else there exists an infinite 
path in decision tree. If deg(rj) = and lc(r.;) > 0, the algorithm works the same as the classical 
Euclidean algorithm for integers. If deg(rj) > 0, then since < rj + i(n) < rj(n) for n ^> 0, we have 
three cases for the next step: 

1. deg(r m ) < deg(rj) 

2. deg(rj + i) = deg(rj) and lc(rj + i) < lc(rj). After finitely many iterations, one will either have 
lc(r i+ i) = lc(rj), or deg(r m ) < deg(n). 

3. deg(r i+ i) = deg(rj) and lc(r m ) = lc(rj). In the next step, deg(r i+2 ) < deg(r m ). 

So the algorithm terminates after finitely many steps. □ 

Tracing through a natural generalization of the Euclidean algorithm, we can get u, v G QP >0 
such that uf + vg = ggcd(/, g). More generally for /1, . . . , fk G QP>o we have 

Corollary 4.4. Let f\,...,fk £ QP>o> d = gg cc K/i, • • • 1 fk)- Then d G QP>o an d there exists 
u±, . . . , Uk G QP>o with ggcd(ui, . . . , = 1 such that 

H h f k u k = d. 

Remark 4.5. This implies, in particular, that every finitely generated ideal of QP >0 i s principal. 
It is peculiar that in fact QP >0 is n °t a Noetherian ring. For example, take Xd to be the function 
such that Xd{x) = 1 if d\x, and otherwise. Then / = (x2i X4i X8j • • • ; X2 n , ■ ■ ■) is not a finitely 
generated ideal. 

Recall that for any M G Z fcxs , there exist unimodular matrices U, V, i.e., determinant ±1, such 
that UMV = D = (diag(<ii, . . . , d r , . . . , 0)|0) with di\di+i and D is called the Smith normal form 
of M. By generalized division over QP >0 (Algorithm [2]) and Corollary 14.41 we can generalize the 
classical Smith normal form theorem as follows. 
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rkxs 



such that D(n) is 



Theorem 4.6. For any M £ (QP >0 ) ; define a matrix function D : Z 
the Smith normal form of M(n). Then we have D € (QP >0 ) 
V 6 (QP >0 ) SXS such that U(n),V(n) are unimodular for n 3> and UMV = D. We call this 
matrix function D the generalized Smith normal form of M . 



^ kxs and there exists U € (QP >0 ) fcxfc 



Before the proof, let us see a simple example for generalized gcd and generalized Smith normal 
form. 

Example 4.7. Let / = 2, g = x. Denote d = ggcd(/, g), namely d{n) = gcd(2,n). 

Put T = 2 and do(x) = 2, d\{x) = 1, uq(x) = 1, ui(x) = —x, vo(x) = 0, v\(x) = 1, we have 

d(n) = di(m) = Ui(m)f{n) + Vi(m)g(n) for all n = Tm + i and i = 0,1. 

2 3a; + 2 



For A 



x x 3 + 2x 



let U(x) = [Uq(x),Ui(x)], where 



U n (x) 



1 

-x 1 



Ul{x) 



—x 1 
-2x - 1 2 



1 -(3x + l) 
1 



and V(x) = [Vo(x), Vi(x)], where 
V (x) 

and D{x) = [Dq{x),D\(x)\, where 

D (X) : 



Vx(x) 



-(8x 3 



6x 2 
1 



5x + 3) 



2 

8x 3 





6x 2 + 2x 



£>i(z) 



1 







16x 3 + 12x 2 + Ax + 1 



We have £7j(m)yl(n)y(m) = Di(m) is the Smith normal form for A(n) for n ^> where n = 2m + i 
and i = 0,1. 

The following proof is based on the proof for classical Smith normal form in [New] . 
Lemma 4.8. Let a\, c*2, ■ ■ ■ , dk € QP >0 with ggcd(ai, o<2, ■ ■ ■ , os/-) = dk- Then there is a matrix 



W k € (QP 



\fcxfc 



with first row [ct\, a2, ■ ■ ■ , and determinant dk- 



Proof. By induction on k. For k = 1, trivial. Now suppose the lemma is true for k = n — 1, 
where n > 2, and let Wk-i € (QP^q)^ ^ be a matrix with first row [a\, ot2, ■ ■ ■ , and 

determinant dk-i = ggcd(ai, Q2, • • • , afc-i)- Since 

4 = ggcd(ai,a 2 , •••,«*;) 

= ggcd(ai,a 2 , . . . ,a fc _i),a fc ) 

= ggcd(dfc_l,Q!jfc), 

by Corollary there exist p, a G QP>o such that pdk-i — oak = dk- Put 

/ 



W, 



Then we have W k e (QP >>0 ) /cxfc with first row [a±, ct2 



\dfc-l 



dfc-1 











,a k ] and det(H / fc ) 



pdk-i-aau = dk- □ 
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Proof of Theorem\l6\ Write A = (ay) ^Og (QP >>0 ) fexs . If an ^ 0, then switch rows and 
columns so that this is true. Without loss of generality, we may assume that an divides all elements 
in the column. If not, let d = ggcd(an, . . . , a^i). By Corollary 14.41 there exist u\, . . . , Uk € QP>o 
with ggcd(ni, . . . , Uk) = 1 such that u\a\\ + • • • + UkOki = d. By Lemma l4~8| there exists a matrix 
W G (QP >0 ) fcxfc with first row [u±, . . . ,Uk] and determinant 1. So we have A' = (a^) = WA such 
that a' u divides all elements in the column. Suppose an divides all elements ay. First, by the 
same process as above, we can assume that an divides all elements in the first row and column. 
By elementary row and column operations, all the elements in the first row and column, other than 
the (1, 1) position, can be made zero. If there is some element ay, i,j ^ 1 is not divisibleby an, 
add column j to column 1 and repeat the previous step to replace an with one of its divisor which 
divides ay. Thus we must finally reach the stage where the element in the (1, 1) position divides 
every element of the matrix and all the other elements of the first row and column are zero. Repeat 
the entire process with the submatrix obtained by deleting the first row and column. □ 



5 Application of generalized division in the computation of some 
generalized Ehrhart polynomials. 

We can see that the algorithm we give in the proof of Theorem 11.11 is not very efficient. In this 
section, by the theory of generalized division developed in the previous section, we are able to 
compute some special cases more efficiently. 



5.1 Generalized Popoviciu formula. 

First recall the classical Popoviciu formula. Let a and b be relatively prime positive integers. Then 
the number of nonnegative integer solutions (x, y) to the equation ax + by = n is given by the 
formula (jl.ip . Now consider the equation 

a(n)x + b(n)y = m(n) 

where a(n),b(n),m(n) are integeral polynomials of n. By the theory of generalized division, the 
above formula for the number of nonnegative integer solutions P{ ai (n),a 2 (n)}( m ( n )) can De easily 
generalized to the following 

Proposition 5.1. Suppose ggcd(a(n), b(n)) = 1, then 

m(n) ( ai(n) _1 m(n) 1 f a 2 (n)- 1 m(n)\ 

P{a l{ n)^ { n)}{m{n)) = - — - -—r + 1 



0,1(71)0,2(11) I a2(n) J [ ai(n) 

where a\(n)ai(n) 1 + 02(71)02(11) = 1. In particular, P{ ai ( n ) jCl2 ( n )} (m(n)) G QP >>0 . 

If ggcd(a(n), b(n)) = d(n) ^ 1, we can simply divide it from both sides and reduce to the case 
of ggcd(a(n), b(n)) = 1. 

Then as mentioned in the introduction, we can consider the number of solutions (x, y, z) £ Z> 
to the matrix equation (jl.2p where the Xi and yi are fixed positive integers and Xi + \yi < Xiyi + i for 
i = 1,2. Write Yy = Xiyj — Xjyi. We assume that gcd(Y"i2, V13, Y23) = 1, so that there exist integers 
(not unique) /y , gy such that 

gcd(/i 2 y"l3 + #12*23, Yl2) = 1, gcd(/i 3 yi2 + 013*23,^13) = 1, gcd(/23>l3 + 923*12, *23) = 1- 
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Now define two regions Qj = \(x,y) I — < - < ^^-} for i = 1,2. Then if m = (7711,7712) € Z 2 is 
in the positive span of the columns of the matrix in (|1.2|) . there exist the following Popoviciu-like 
formulas |Xu} Theorem 4.3] for t(m\A), the number of solutions of (|1.2p . 
When 771 = (7711, 77i 2 ) T € f^i PI Z 2 , 

., I a \ m 2 x 1 -m 1 y 1 ( (/i 2 *i3 + 3i2*23)~ 1 ( m 2(./i22:i + 312^2) - mi(/i 2 yi + 3122/2)) 
H m M = 1 V 

r 12 r 13 I r 12 

[ (/i3*i2 + fli3^23)~ 1 ( m 2(/i3^i + 313^3) ~ rn 1 (f 13 y 1 + 31333)) \ 

I ^13 J 

When 77i = (mi,77i2) T G ^2 n Z 2 , 

miy 3 - m 2 y 3 ( (723*13 + 323*12)~ 1 ( ?71 l(/23^3 + 323^) - m 2 (/232/3 + 3232/2)) 



t(m|^) 



*23*i 3 I *23 

(/l3*12 +3l3^23)~ 1 ( ?Tl l(/l3^1 +313^3) -m 2 (/l32/l +3132/3)) 



+ 1, 



*i 3 

where /i 2 , 312, /13, 313, /23 and 523 G Z satisfy 

gcd(/i 2 li3 + 312*23, Yl2) = gcd(/i 3 li2 + 313*23, Y 13 ) = gcd(/ 23 *13 + 323*12, *23) = 1 

and 

(/l2*13 + 3l2*^23) _1 (/l2*"l3 + 3l2*2 3 ) = 1 (mod Y 12 ), 
(/l3*12 + 3l3*"23) _1 (/l3*"l2 + 3l3*2 3 ) = 1 (mod Y 13 ), 
(.723*13 + 323*12 ) _1 (/23*13 + 323*12) = 1 (mod Y 23 ). 

Notice that in the above formula, everything can be directly generalized to the ring Z[x] using 
generalized division and generalized GCD. In other words, we can replace the rcj, 7/3, and nii by 
polynomials in 71 in such a way that for all values of n, the condition gcd(Yi 2 , *i3, 123) = 1 always 
holds. For example, consider the system 

2n + 1 3n + l 7i 2 \ l X \ _ ( 3ti 3 + 1 
2 3 71 + 1/ I V J ~ \3n 2 +n-l 

Then for n>0, we have that 

3 3ti 2 + ti-1 71 + 1 

3n + 1 3n 3 + 1 n 2 ' 

so the number of solutions (x, y, z) is counted by the formula of t{m\A) for 771 G £l 2 and it is easy 
to see that t(m\A) G QP >0 - 



5.2 Dimension two. 

For polytopes in dimension two, there is an efficient algorithm for computing the number of lattice 
points using continued fractions. See [Bar I Chapter 15] for the algorithm in the case of non- 
parametrized two dimensional polytopes. 

By the generalized division theory, we can generalize this algorithm to the case of a polytope 
whose vertices are given by polynomials. Notice that the only difference is the generalization 



17 



of continued fractions: for any /, g € Z[x], we expand f/g as a continued fraction. Then by 
Theorem 14.31 f° r n^> 0, the terms in the expansion are quasi-polynomials and the number of terms 
does not depend on n. 

We use the following notation for continued fractions. First, set [a] = a and in general, we set 



[o ;ai, ...,a k ]=a + 



[ai;a 2 , . . . , a k ] 



Example 5.2. For n > 4, we have 



n~ 



2n + 1 



[m — 1; 1, 3, m] n = 2m 
[m — 1; 3, 1, m — 1] n = 2m — 1 



And for the cone generated by (0,1) and (2n + l,n 2 ), we have the following efficient algorithm of 
decomposition into prime cones. 

(2n + l,n 2 ) Q,™,) (2n + l,n 2 ) 











1 (l,m-l) / (l,m-l) 
+ 





(4,4m - 1) 















+ 




(4,4m - 1) 
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